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1.  JBIBflMfflQH 


This  is  the  Tenth  Quarterly  Progress  Report  prepared  ooverlng  the  Georgia  Tech 

Researoh  Program  in  Fully  Distributed  Processing  Systems  (FDPS). 

a,  ^Program  Description. 

The  Georgia  Tech  Research  Program  in  Fully  Distributed  Processing  Systems 
is  a  comprehensive  investigation  of  data  processing  systems  in  which  both 
the  physical  and  logical  components  are  extremely  loosely  coupled  while 
operating  with  a  high  degree  of  oontrol  autonomy  at  the  component  level. 
The  definition  of  the  specific  olass  of  multiple  computer  systems  being 
investigated,  and  the  operational  characteristics  and  features  of  those 
systems  is  motivated  by  the  desire  to  advance  the  state-of-the-art  for  that 
class  of  systems  that  will  deliver  a  high  proportion  of  the  benefits 
currently  being  claimed  for  distributed  processing  systems.  The  scope  of 
individual  topics  being  investigated  under  this  program  ranges  from  formal 
modeling  and  theoretical  studies  to  empirical  examinations  of  prototype 
systems  and  simulation  models.  Also  included  within  the  scope  of  the 
program  are  areas  suoh  as  the  utilization  of  FDPS’s  and  their  interaction 
with  management  operations  and  structure. 

b.  Program  Support.  ' 


The  principle  support  for  the  program  is  a  Selected  Research  Opportunity 
contraot  from  the  Office  of  Naval  Research;  however,  there  are  a  number  of 
other  sources  of  funding  which  also  support  the  program.  A  list  of  the 
currently  active  contracts  and  grants  is  given  below. 

Title:  "Research  on  Fully  Distributed  Data  Processing  Systems" 

Funding  Agency:  Office  of  Naval  Researoh  (ONR) 

Contract  Number:  N00014-79-C-0873 

GIT  Project  No. :  G36-643/336 

Principle  Investigator:  Philip  H.  Enslow,  Jr. 

Title:  "Evaluation  of  Distributed  Control  Models" 

Funding  Agenoy:  U.S.  Air  Force  Rome  Air  Development  Center  (RADC) 

Contract  Number:  F30602-78-C-0120 
GIT  Project  No, :  G36-654 

Principle  Investigator:  Philip  H.  Enslow,  Jr. 


Title:  "System  Support  Capabilities  for  Fully-Distributed 
Loosely-Coupled  Processing  Systems" 

Funding  Agenoy:  O.S.  Air  Foroe  Rome  Air  Development  Center  (RADC) 
Contraot  Number:  F30602-81-C-0249 
GIT  Projeot  No.:  G36-659 

Principles  Investigator:  Philip  H.  Enslow,  Jr. 

Title:  "Theory  of  Systems  of  Asynchronous  Parallel  Processors" 

Funding  Agenoy:  O.S.  Army  Researoh  Of floe  (ARO) 

Contraot  Number:  DAAG29-79-C-0155 
GIT  Projeot  Number:  036-638/332 
Principle  Investigator:  Nanoy  A.  Lynch 
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Title:  "Complexity  and  Computability  for  Distributed  Data  Bases" 
Funding  Agency:  National  Solenoe  Foundation  (NSF) 

Contract  Number:  MCS-7 924370 
GIT  Project  Number:  G36 -652/340 
Principle  Investigator:  Nancy  A.  Lynch 


2.  ORGANIZATION  AND  STAFFING 


Faculty 

DeMillo,  Richard  A.  -  Professor 
Enslow,  Philip  H.  Jr.  -  Professor 
Grlffeth,  Nancy  A.  -  Assistant  Professor 
Jensen,  Alton  P.  -  Professor 
LeBlanc,  Richard  J.  -  Assistant  Professor 
Livesey,  Jon  -  Assistant  Professor 
Lynch,  Nancy  A.  -  Associate  Professor 
McKendry,  Martin  S.  -  Assistant  Professor 
Miller,  Raymond  -  Professor 
Underwood,  William  -  Assistant  Professor 


McDonell,  Sharon  -  Administrative  Secretary 
Myers,  Jeanette  -  Research  Scientist 
Pinion,  Nancy  -  Part-time  Secretary 
Monglovi,  Roy  -  Research  Technologist  I 


Students 

There  are  approximately  30  students  working  on  various  projeots  in  the  FDPS 
Research  Program.  Of  these,  12  are  in  the  Ph.D.  program,  and  5  are 
preparing  their  M.S.  Thesis  on  topics  in  FDPS. 


3.  flinmr  mmbabch  pi 


The  speolflo  research  projeots  have  been  organised  into  the  major  areas 
identified  in  the  basic  program  proposal. 


A.  IhflttttfclfliiL  JUBA  JEfiEML  ttldlM 
A. 3  Reliable  Systems 

A. 4  Time  Performance  of  Distributed  Systems 

A. 5  Audit  Algorithms 

A. 6  Tloket  Systems 

A. 9  Theory  of  Distributed  Databases 

A. 16  Stoohastlo  Synchronisation 
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A. 17  Research  Allocation  In  a  Failure-Prone  Environment 
A. 18  Multilevel  Atomloity 

A. 19  Formal  Semantics  and  Specification  of  Distributed  Systems 
A. 20  Nested  Transactions  with  Aborts 


b.  Physical  interconnection  and.  J&UamcElnt 

B.2  Local  Networking  in  Fully  Distributed  Processing  Systems 


C.  Distributed  Operating  Systems 

C.1  Decentralized  and  Distributed  Control 

C.2  Resource  Allocation  and  Work  Distribution  in  an  FDPS 

C.M  Local  Operating  System 

C.5  Communications  Support  for  Distributed  Systems 
C.8  Distributed  Software  Tools 
C.9  Command  Languages  in  an  FDPS 

C. 10  Distributed  Operating  System  Implementation 

D.  Distributed  Data  Bases 

D.  1  Concurrency  Control  in  Distributed  Database  Systems 
D.3  Implementation  of  the  Audit  Algorithm 

D.4  User  Interfaces  to  Database  Systems 


e.  fault-Iolorance 


f.  special  Hardware  in  Support  £BES. 


G.  Application  fit  Distributed  ProCOBBlM 


H.  System  Design  Methodologies 

H.2  Coordinating  Large  Programming  Projects 


I.  System  Utilisation 

1.1  A  Language  for  Distributed  Programming 

1.2  System  Implementation  Language  Development 

1.3  Experiments  with  a  Distributed  Compiler 


J.  Security 

J.1  Process  Structures 
J.2  System  Seeurlty 
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Km  flatu  fife 

l.  EralugUoa  sad.  fagpirlflQa 

L.  1  Simulation  of  Distributed  Algorithms  (Grlffeth,  Lynch) 
M.  PDFS  Testbed 

M. 1  Establishment  of  PDPS  Testbed  Facility 

M.3  Fully  Distributed  Operating  System  Simulation  Testbed 


4.  SUMMARY  OF  PROGRESS 


A. 3  Reliable  Systems  (Lynch,  Pi sober,  Fowler,  Lamport,  Merritt) 

A  new  Byzantine  Generals  algorithm  was  devised,  with  better  performance 
than  previously  known  algorithms,  In  terms  of  amount  of  message  traffic. 
The  number  of  rounds  Is  also  considerably  smaller  than  in  previous 
algorithms  with  good  message  performance. 

The  lower  bound  proof  for  number  of  rounds  In  an  environment  allowing 
arbitrary  authentication  capabilities,  has  been  considerably  refined  and 
clarified. 


A.4  Time  Performance  of  Distributed  Systems  (Lyneh,  Pisoher,  Lazowska, 
Sohdnbage) 

No  significant  progress  to  report. 


A. 5  Audit  Algorithms  (Grlffeth,  Pisoher,  Lynch) 

The  paper,  "Global  States  of  a  Distributed  System”,  was  acoepted  for  a 
special  issue  of  Transactions  on  Software  Engineering  (May,  1982).  Also, 
work  is  in  progress  on  analysis  of  some  significant  special-case  systems. 


A. 6  Tieket  Systems  (Pisoher,  Grlffeth,  Qulbaa,  Lynch) 

The  tioket  system  simulation  has  been  modified  to  reduce  the  varianoe  in 
results  and  generalized  to  allow  returns  and  reallocation  of  tickets. 
Initial  tests  have  been  run.  Purther  modifications  are  planned  to  test 
limiting  oases. 


A.9  Theory  of  Distributed  Databases  (Lynoh,  Grlffeth) 
No  significant  progress  to  report. 


Page  -4 


GIT  FDPS  Research  Program 


Quarterly  Prog  Report  10 


A. 16  Stochastic  Synchronization  (DeMillo,  R.  Miller,  Lipton) 

No  significant  progress  to  report. 

A. 17  Resource  Allocation  In  a  Failure- Prone  Environment  (Fisoher,  Lynch, 
Burns,  Borodin) 

No  significant  progress  to  report. 


A. 18  Multilevel  Atomicity  (Lynoh) 

Some  work  is  being  done  to  Integrate  the  multilevel  atomicity  ooncept  with 
related  work  on  nested  transactions. 


A. 19  Formal  Semantics  and  Speoifloation  of  Distributed  Systems  (Lynoh,  Stark) 

Equivalence  has  been  proved  for  three  natural  definitions  for  the  class  of 
possible  behaviors  of  distributed  systems. 


A. 20  Nested  Transactions  with  Aborts  (Lynoh,  Leskov) 

Formal  correctness  oonditions  for  nested  transaction  systems,  and 
correctness  proofs  for  implementations  using  locking,  are  in  process  of 
development. 


B.2  Local  Networking  in  FDFSs  (Bnslow,  Myers,  Brundette,  Hutchins,  Arius) 

A  12-1/2  hour  video  Net/One  Programing  class  created  by  Ongermann-Bass 
was  taken  by  members  of  this  project.  This  course  desorlbed  in  detail 
Net/One  operation  and  the  recently  released  software  development  support 
package. 

We  are  waiting  for  the  delivery  of  a  new  component  for  the  looal  network, 
an  NCF-2,  which  replaces  the  MCZ.  (The  MCZ  is  no  longer  supported  by 
Ungermann-Bass) .  The  NCF-2  includes  the  software  development  support 
package,  support  for  the  C  programming  language  and,  due  to  an  IEEE-448/79 
interface  to  the  local  network,  reduoes  downloading  time  from 
approximately  90  seconds  to  19  seconds  per  board. 


C.1  Decentralized  and  Distributed  Control  (Ena low,  LeBlanc,  Saponas) 

A  technical  report  concluding  Tim  Saponas* s  research  in  this  area  is  ready 
for  distribution.  This  report  analyzes  in  detail  the  factors  contributing 
to  the  particular  evaluations  of  eaoh  of  the  oontrol  models  oovered  in  a 
previously  distributed  technical  report. 
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C.2  Resource  Allocation  and  Work  Distribution  in  an  FDPS  (Bnalow,  Sharp) 

A  technical  report  concluding  Don  Sharp's  research  in  this  area  Is  ready 
for  distribution.  Results  of  simulation  experiments  indicated  that  the 
best  criterion  for  work  distribution  is  to  minimize  communication  between 
processes  represented  as  nodes  of  a  task  graph. 


C.4  Local  Operating  System  (Livesey,  LeBlano,  MoKendry,  Myers,  Allchin, 
Fukuoka,  Maooabe,  Pitts,  Spafford) 

Work  continues  on  the  identification  and  research  of  those  design  concepts 
related  to  distributed  operating  systems.  This  includes  atomicity, 
consistency,  recovery,  fault  tolerenoe,  interprocess  communication,  file 
systems,  etc.  Of  special  importance  is  the  area  of  distributed  databases. 
Due  to  the  amount  and  distribution  of  information  required  by  a 
distributed  operating  system,  we  are  looking  into  building  a  distributed 
database  as  an  intrinsic  part  of  the  operating  system. 

Project  C.10  has  been  created  to  attempt  a  "first  try"  implementation  of 
this  combination  of  data  base  and  operating  system. 


C.5  Communications  Support  for  Distributed  Systems  (Enslow,  Skowbo) 

The  formal  project  proposal  is  essentially  complete  and  is  currently  being 
reviewed,  revised,  and  extended.  The  specification  and  design  of  tools 
for  an  experimental  evaluation  of  proposed  alternatives  for  communications 
support  is  also  in  progress. 


C.8  Distributed  Software  Tools  (Myers,  Livesey,  Hopkins,  Fox) 

Work  continues  on  the  Implementation  of  Distributed  Software  Tools  and 
preparation  of  a  technical  report  describing  our  experiences. 


C.9  Command  Languages  in  an  FDPS  (Badre,  Myers,  Greene) 

During  this  quarter,  the  main  effort  has  been  direoted  toward 
characterizing  the  user,  his  requirements,  and  his  needs.  To  accomplish 
this  in  part,  user  aotlvity  has  been  monitored  and  these  data  processed  to 
determine  how  people  are  using  the  ourrent  facilities.  We  are  studying 
what  oonnanda  are  being  used  and  in  what  general  class  of  processing  these 
conmanda  are  used  (e.g.  text  manipulation,  program  development,  etc.). 
Also,  the  commands  available  in  the  same  system  have  been  examined  to 
determine  what  these  commands  funotionally  provide  to  a  user.  The 
information  gathered  from  these  approaches  is  being  used  as  a  strong  basis 
for  the  friendly  user  interface  that  is  the  goal  of  this  work. 


C.10  Distributed  Operating  System  Implementation  (MoKendry,  Allohln,  Thlbault, 
Maooabe) 

Realising  that  there  remain  many  unsolved  problems  and  open  areas  of 
research  related  to  distributed  operating  systems,  we  believe  that  it 
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would  be  beneficial  to  begin  an  implementation  based  on  what  we  currently 
know  that  we  can  do.  We  have  named  our  prototype  operating  system 
"CLOUDS",  an  acronym  for  "Coalescing  Local  Operating  Systems  Under 
Decentralized  Supervision."  The  descriptor  "Coalescing"  is  an  Important 
one.  CLOUDS  will  be  capable  of  stand-alone  operation,  but  when  networked 
with  other  CLOUDS,  will  naturally  oome  together  or  "coalesce"  into  one 
distributed  operating  system. 

Current  efforts  are  centered  around  designing  a  scaled  down  version  of 
such  an  operating  system  for  the  first  implementation. 


D.1  Concurrency  Control  in  Distributed  Database  Systems  (Griffeth,  Llvesey, 
Lynch) 

No  significant  progress  to  report. 


D.3  Implementation  of  the  Audit  Algorithm  (Griffeth,  Llvesey,  Lynoh) 
No  significant  progress  to  report. 


D.4  User  Interfaces  to  Database  Systems  (Griffeth) 

Several  interfaces  have  been  completed  for  the  PRIME  relational  database. 
They  are:  (1)  a  relational  algebra  interface,  (2)  a  relational  calculus 

interface,  and  (3)  a  network  interface.  Preliminary  tests  on  the 
effectiveness  of  these  interfaces  will  begin  next  quarter. 

A  project,  "Distributed  Database  Algorithms",  will  be  funded  by  NSF  for 
the  period  July  1982  -  June  1984. 


H.2  Coordinating  Large  Programming  Projeots  (Enslow,  Smith) 

A  number  of  additional  data  sources  were  identified.  A  tentative  metric 
for  the  quality  of  communication  activities  during  large  software 
development  was  proposed  and  is  currently  under  investigation. 


1.1  A  Language  for  Distributed  Programming  (LeBlano,  Maooabe) 

Design  work  has  been  completed;  implementation  and  evaluation  are  in 
progress.  A  paper  was  submitted  to  the  Third  International  Conference  on 
Distributed  Computing  Systems. 


1.2  System  Implementation  Language  Development  (LeBlano,  MoKendry,  Wilkes) 

Implementation  of  a  Pascal  compiler,  using  the  code  generator  previously 
developed  under  this  project,  is  nearly  oomplete.  Design  of  extensions 
for  system  implementation  support  is  in  progress.  Implementation  should 
begin  during  next  quarter. 
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I. 3  Experiments  with  •  Distributed  Compiler  (LeBlano,  J.  Miller) 

Work  by  Miller  has  confirmed  and  considerably  improved  earlier  results. 
Design  for  balanced  message  flows  was  found  to  be  crucial  for  best 
performance.  A  paper  was  submitted  to  the  Third  International  Conference 
on  Distributed  Computing  Systems. 

J. 1  Prooess  Structures  (DeMillo,  Llpton,  B.  Miller,  Merritt,  Thomas) 

No  significant  progress  to  report. 

J.2  System  Security  (Livesey,  Davida,  DeMillo) 

No  significant  progress  to  report. 

L. 1  Simulation  of  Distributed  Algorithms  (Griffeth,  Lynch) 

No  significant  progress  to  report. 

M. 1  Establishment  of  FDPS  Testbed  Paoility  (Myers,  Monglovl,  Fox) 

No  significant  progress  to  report. 

M.3  FDOS  Simulation  Testbed  (LeBlano,  Saponas,  Myers) 

No  significant  progress  to  report. 


.  TRAVEL  TO  THE  FDPS  PROGRAM 


Date  of  Trip;  25-27  January,  1982 
Ipdlvldual(a)  Traveling;  Richard  LeBlanc 
Itinerary;  Albuquerque,  New  Mexioo 

Purpose;  Attend  ACM  Principles  of  Programming  Languages  Symposium 

Date  of  Trip;  February,  1982 
individual^)  Traveling;  Nancy  A.  Lynoh 

Itinerary;  Brandeis  University,  Northeastern  University,  Boston  University 
Purpose;  Speak  about  new  Byzantine  Generals  algorithm. 
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6.  .VISITORS 


Dates  of  Visit:  February,  1982 
Visitor:  Bharat  Bhargava 
-Contact:  Nancy  A.  Lynch  (at  NIT) 

Purpose:  Discussions  about  correctness  proofs  of  concurrency  control 

algorithms,  and  about  reliability  properties  of  distributed  algorithms. 


7.  swdswm 


Author ( si :  John  A.  Miller  and  Richard  J.  LeBlanc 
Title:  Distributed  Compilation:  A  Case  Study 
Type:  conference  paper 
Status :  submitted 

Author(s) :  Arthur  B.  Maccabe  and  Richard  J.  LeBlanc 

Title :  The  Design  of  a  Programming  Language  Based  on  Communication  Networks 
Type:  conference  paper 
Status :  submitted 

Author(s) :  Michael  J.  Fischer,  Nancy  D.  Griffeth,  and  Nancy  A.  Lynch 
Title:  Global  States  of  a  Distributed  System 
Type:  journal  paper 

accepted  for  publication  in  Transactions  on  Software  Engineering 
Publ.  Date:  May,  1982 

Author(s) :  Nancy  A.  Lynch 
Title:  Multilevel  Atomicity 
Type:  conference  paper 

Status:  accepted  by  Principles  of  Database  Systems  Conference 
Publ.  Date:  March,  1982 

Author(s) :  Nancy  A.  Lynch,  Michael  J.  Fischer,  and  Robert  Fowler 
Title :  A  Simple  and  Efficient  Byzantine  Generals  Algorithm 
Type:  conference  paper 

Status:  submitted  to  IEEE  Symposium  on  Reliability  in  Distributed  Software 

and  Database  Systems. 

Author(s):  Richard  A.  DeMillo,  Nancy  A.  Lynch,  and  Michael  Merritt 
Title :  Cryptographic  Protocols 
Type:  conference  paper 
Status:  accepted  by  SIGACT 
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